Distributed Vector Architecture : Beyond a Single Vector - IRAM

integration of memory on the same die as the processor (IRAM) has the potential to offer unprecedented bandwidth that can be exploited efficiently by vector processors. However, real-world scientific vector applications with their very large memory requirements and their poor locality, would easily overflow any single IRAM device. In this environment, traditional approaches such as caching or paging generate considerable traffic, diminishing the performance advantage of processor-memory integration. To exploit the full potential of IRAM in the realm of large-scale scientific computing , we propose a DIstributed Vector Architecture (DIVA), that uses multiple vector-capable IRAM nodes in a distributed shared-memory configuration. The advantages of our approach are twofold: (i) we speed up the execution of the vector instructions by parallelizing them across the nodes, (ii) we reduce external traffic, by bringing computation to data rather than data to computation. We dynamically map the computation of individual vector instructions on nodes to coincide, to the extent possible, with the corresponding data in memory. As an implementation, we propose a mechanism to assign at run-time elements of the architectural vector registers on nodes, using the layout of data in memory as a blueprint. Using traces of vector supercomputer programs we demonstrate that DIVA often generates considerably less external traffic compared to single or multiple-node alternatives that are based solely on caching or paging. Considerable performance gains are then possible because of DIVA's inter-node parallelism.